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ABSTRACT 

Attitude measures frequently produce distributions of Item scores 
that attenuate Interltem correlations and thus also distort 
findings regarding the factor structure underlying the items. An 
actual data set involving 260 adult subjects' responses to 55 
items on the Love Relationships scale is employed to Illustrate 
empirical methods for identifying such items during test 
construction or test validation. Two methods for identifying 
these items are proposed, and it is noted that examination of 
item descriptive statistics may not successfully substitute for 
an empirical method for identifying potentially troublesome 
items . 
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The first factor analytic method was the principal 
components model originally conceptualized by Pearson (1901) and 
later developed by Kotelling (1933)- The merits of few techniques 
have been debated as heatedly as have the merits of factor 
analysis (Royce^ 1980; Tryon^ 1979). Cronkhite and Liska (1980^ 
p. 102) are particularly ascerbic in their comment that it is 
apparently 

so easy to capture college students to use the 
scales to rate the sources^ so easy to submit 
those ratings to factor analysis^ so much fun to 
name the factors When one's research assistant 
returns with the computer printout^ and so 
rewarding to have a guaranteed publication with no 
fear of nonsignificant results that researchers^ 
once exposed to the pleasures of the factor 
analytic approach^ rapidly become addicted to it. 
Similarly^ Sax (1979^ p. 80) notes that 

the relative simplicity o£ this discriminative 
[ i .e . ^ correlational 1 approach has? appealed to 
many graduate students [and faculty! who^ once 
they have mastered the technique of computing 
correlations or other statistics^ have used these 
methods without due regard for the value of their 
research proposals. 
Notwithstanding these criticisms^ it is clear that some 
factor analytic applications have proven very useful to 
behavioral scientists. Firsts factor analysis can be usefully 
applied as a prelude to analysis of variance techniques in 
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experimental studies (Morrow & Franklewlcz^ 1379}. Mcculloch and 
Thompson (1982) report such a study In which factor analysis was 
employed to conserve degrees of freedom as a function of the data 
reduction features of the method. These applications may be 
especially useful if factor scores are computed in their non- 
Zscore form (Thompson^ 1983). second^ factor analysis can be 
employed in substantive investigations to minimize distortions 
introduced by measurement error (Thompson^ 1982^ pp. 2-4). Thirds 
the methods can provide potent assistance in theory building^ as 
Guilford (1959) and others (cf. Thompson^ 1980) have 
demonstrated. 

But factor analysis has been shown to be particularly useful 
in instrument construction or in investigating Instrument 
validity. Thus Nunnally (1967, p. 100) notes that some 
researchers have referred to construct validity as "factorial 
validity." Various examples of this application are available 
(cf. Thompson & Borrello, 1986). Thompson and Plttg (1981/82) 
have proposed confirmatory factor rotation as one empirical 
method for evaluating how adequately actual and expected results 
correspond, confirmatory factor extraction methods (Joreskog, 
1969) can also be employed to evaluate Instrument validity (cf. 
Thompson, Wisbber & Berenson, 1987). 

The purpose of the present paper is to present two methods 
for identifying items that may distort the factor structure 
underlying item responses. It will be argued that these methods 
may be particularly helpful in the test construction of attitude 
measures. In order to make the discussion concrete, an actual 
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data set (Borrello & Thompson^ 1986) will be employed to provide 
heuristic examples. The data were provided by 260 adult subjects 

who reacted to the 55 attitude statements on the Love 
Relationships Scale, a measure o£ attitudes about love and about 
love relationships . 

Theoretical Background for the Methods 
The principal component model is among the simplest of the 
factor analytic models. The method assumes no measurement error 
and that all the variance of each factored entity is relevant to 
the analysis. Unlike various other models, the principal 
components method does not require Iterative estimation of the 
communalltles of each variable, due to the assumption of no 
measurement error. In addition to its simplicity, the model has 
the advantage that factor scores based on model results can be 
directly calculated rather than estimated (Gorsuch, 1983, p. 
122). Furthermore, if the factors are uncorrelateJ due to 
orthogonal rotation, the factor scores produced using the model 
are also perfectly uncorrelated; this will not be the case when 
certain other models are employed. 

Most exploratory factor extraction procedures, e.g., 
models using principal factors, image, alpha, or canonical 
extraction methods, are variants of the principal components 
method. Even in the presence of measurement error, Monte Carlo 
studies (Gorsuch, 1983, p. 123) Indicate that when the number of 
factored entlti^^s (usually variables) is large, the methods 
produce very similar results . As Thompson (1986, p. 3 ) explains. 
This is because the number of entries on the 
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diagonal o£ the correlation [or other association 
Index] matrix becomes rapidly smaller relative to 
the total number of matrix values as the number of 
factored entitles (v) Increases (2/4 If v=:2; 3/9 
If v=:3; 4/16 If v=:4; etc.). 
Although principal components methods are typically employed 
to extract factors from the correlation matrix^ factors can also 
be extracted from other matrices as well. For example^ factors 
can be extracted from a matrix consisting of covarlances off the 
matrix diagonal and the variance of each variable on the matrix 
diagonal. The factors extracted from a varlance-covar lance 
matrix^ however^ are not "scale free." That Is^ the factors in 
such a case are Influenced both by the patterns of associations 
among the variables (i.e.^ covarlance coefficients) and by the 
variabilities of the variables (i.e., the variances). In general 
this is not desirable in social science research, since typically 
the metrics of ability or attitude measures have little intrinsic 
meaning. For example, a set of test scores may have a standard 
deviation of 5.0. If the scores are multiplied by 2.0, the 
standard deviation of the "new" scores will be 10.0, but nc 
meaning will have been lost or changed. The metrics of most 
scales in the non-physical world are arbitrary and have been left 
to chance or have been selected by the researcher for 
convenience . 

The apparent implication is that usually one would prefer 
factors extracted from a correlation matrix, since these factors 
are not influenced by arbitrary scale characteristics. However, 
in reality correlation coef f icients are indeed affected by 



differences In distribution shapes^ Including differences In 
variability. This Is taught even In elementary statistics texts. 
Thus^ formulas exist to correct correlation coefficients for 
attentuatlon due to restricted variabilities of variables 
(Guilford^ 1965^ p. 480). 

Logically^ i£ factors extracted from a correlation matrix 
associated with a given data set differed from the factors 
extracted from the varlance-covar lance matrix for the same data 
set^ one might learn which variables' variabilities have proven 
problematic as regards establishing an. accurate representation of 
the structure underlying item responses. This comparison might be 
particulatly helpful when the variables are attitude measures^ 
because attitude variables typically Involve skewed distributions 
with somewhat restricted ranges (Kerlinger, 1973^ p. 496). 

Two empirical methods for comparing the factors derived from 
the tv;o matrices are proposed and illustrated employing the 
heuristic data set. Table 1. presents descriptive statistics for 
the 260 subjects' responses on the 55 Love Relationships Scale 
items. Table 2 presents the structure coefficients for factors 
extracted from the correlation matrix^ and rotated to the varimax 
criterion. Table 3 presents the rotated results for factors 
extracted from the varlance-covar lance matrix. 

INSERT TABLES 1^ 2 AND 3 ABOUT HERE. 



Method I ; "Best Fit" Factor Rotation 
In an appendix to a book by other authors^ Kaiser^ Hunka^ 
and Blanchlni (1969) suggested that factors involving the same 



variables could be rotated to positions of "best fit" with each 
other^ and the cosines of the angles among the factors could then 
be computed. These cosines are correlation coefficients, Veldman 
(1967) presents a computer program that performs the necessary 
analysis. Gorsuch (1983^ pp. 283-288) discusses ^;he method. 
Thompson (1986) presents several applications and a partial test 
distribution for these coefficients. 

With respect to the heuristic data from the love attitude 
study^ four of the eight factors extracted from the correlation 
matrix had correlation coefficients with factors from the 
varlance-covar lance matrix greater than 0.95. Two factors from 
the two sources had correlation coefficients across the two 
sources that were greater than 0.80 and less than 0.95. The 
second factor from both the two sources had a correlation 
coefficient of 0.74. The third factor extracted from the 
correlation matrix was appreciably correlated with several 
factors extracted from the varlance-covar lance matrix^ Including 
Factor II (0.65), Factor VI (0.43), Factor VIII (0.38), and 
Factor V ( .35) . 

Gorsuch (1983) has noted that cosines of the angles among 
factored entitles must be consulted in addition to examining 
the cosines among the factors themselves. As Gorsuch (1983, p. 
284) explains, "If the mean cosine Camong the variables] Is low. 
It Is not possible to relate factors at all because the 
relationships among the variables are different." In the present 
e:fample th<^ mean item cosine for the 55 variables waj quite high, 
i.e., 0.94. 
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However^ the Item cosines can still be examined to Identify 
selected items that attentuate the comparability of the factors 
extracted from the two different matrices of association. Six of 
the 55 Items had cosines less than 0.85: Item 21 (0.59)^ Item 38 
(0.70), Item 8 (0.74), Item 37 (0.77), Item 3 (0.77), and Item 44 
(0.79) . 

M echod I I : Comparison of the Reproduced Correlation Matrices 
Each successive factor extracted from the correlation matrix 
using a principal components model accounts for a maximum of the 
variance In the total correlation matrix (R). When all possible 
factors are extracted using this model. I.e., when the number of 
factors equals the number of factored entitles, the factors will 
exactly reproduce the correlation matrix. The formula (Gorsuch, 
1983, p. 77) for using factors to reproduce the "explained" 
portion (R) of the correlation matrix (R) Is: 

R = P P' , 

VxV VxF FxV 

where V equals the number of factored entitles, 

P equals the orthogonal factor structure matrix, 
F equals the number of principal components. 
Thus, In the present example, the 55-by-55 correlation matrix 
would be exactly reporoduced by matrix manipulation of a 55- 
var lable-by-55-factor solution. 

However^ more typical usage of factor analysis Involves an 
effort to "condense" the noteworthy variance In the correlation 
(or the varlance-covar lance ) matrix Into a smaller set of 
factored variables. For example, the researcher might wish to 
condense raw scores of 260 subjects on 55 variables Into factor 
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scores of 260 subjects on each of eight factors. The 
portion (R) of the original correlation matrix (R) that can be 
reproduced from the eight factors could then be estimated by: 

R = P P' . 

55x55 55x8 8x55 

This matrix Is presented for the heuristic data In Appendix A. 

The portion (R*) of the original correlation matrix (R) that 
Is not reproducable from the factors can be calculated (Gorsuch, 
1983, p. 77) by the formula: 

R* = R - R 

55x55 55x55 55x55 

This suggests that the two correlation matrices that would 
be "reproduced" by the factor structure extracted from the 
correlation matrix and by the structure extracted from the 
varlance-covar lance matrix might both be computed and then 
compared to evaluate the effects of variable variability on 
Identified structure. Appendix B presents the "reproduced" 
correlation matrix that was generated for the structure from the 
varlance-covar lance matrix that was presented In Table 2. 
Appendix C presents the difference matrix derived by subtracting 
the "reproduced" correlation matrix from the varlance-covar lance 
factors (Appendix B) from the "reproduced" correlation matrix 
derived using the factors actually extracted from the correlation 
matrix (Appendix A). 

The difference matrix produced in this manner is analogous 
to the "residual" correlation matrix produced by many factor 
analysis statistical packages. This is not actually a correlation 
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matrix. As Gorsuch (1983, p. 143) noteS/ "An error often made Is 
to Interpret l-^e elements of the residual matrix as correlations- 
Despite the fact that the off-diagonal elements look like 
correlations, they are actually covar lances . " 

The non-diagonal entries In this difference matrix would all 
equal zero if the two structure matrices reproduced the same 
correlation matrix. Variables with several large off-diagonal 
entries are the sources of discrepancies between the two 
structures — these variables cause the two solutions to diverge 
due to their variabilities. When the absolute values of the off- 
diagonal entries In this matrix (Appendix C) are summed, the six 
variables with the largest sums are, respectively: iable 38 

(3.53); variable 3 (3.38); variable 44 (2.86); variable 21 
(2.76); variable 36 (2.72); and variable 2 (2.69). When the 
values of the off-diagonal entries in this matrix (Appendix C) 
are squared and summed, the six variables with the largest sums 
^re, respectively: variable 38 (0.38); variable 3 (0.35); 
variable 44 (0.24); variable 21 (0.24); variable 36 (0.23); and 
variable 2 (0.22) . 

Discussion 

Two methods for identifying item:^ that may distort the 
structure of factors extracted from correlation matrices due to 
the items' distributions have been presented. The first method 
involves the use of "best fit" rotation. The second method 
involves the comparison of the "reproduced" correlation matrices 
computed from two structure uiatrices. 

It is noteworthy that both methods identified several of the 
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same Items as being potentially troublesome (e-g.. Items 3^ 21, 
3B, and 44). This suggests that the two methods yield soraewhat 
comparable results. Since the "best fit" rotation method Is more 
widely available to researchers who use Veldman's (1967) computer 
program, this method will often be the method of choice for the 
researcher who wishes to Identify potentially troublesome items. 
However, It may be wise to employ both methods as a cross- 
validation of conclusions regarding potentially troublesome 
. Items . 

It must be emphasised that merely examining descriptive 
statistics such as those presented in Table 1 will not be 
sufficient for Identifying Items whose distributions are 
affecting factor structure, some of the Items Identified In the 
two analyses as being potentially troublesome had distributions 
that were similar to those of Items that did not create 
difficulties. Inspection of descriptive statistics will not serve 
as an acceptable substitute for empirical determination of which 
distributions do or do not affect structure. 

In summary, attitude measures frequently produce 
distributions of item scores that may attenuate interitem 
correlations and thus may also distort findings regarding the 
factor structure underlying the items. An actual data set 
involving 260 adult subjects' responses to 55 items on the Love 
Relationships Scale was employed to illustrate empirical methods 
for identifying such items during test construction or test 
validation. Two methods for identifying these items were 
proposed, and it was noted that examination of item descriptive 
statistics may not successfully substitute for an empirical 
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method for identifying potentially troublesome Items. 
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Table 1 
Descriptive Statistics 



variable 


Mean 


SD 


Kurtosls 


Skevmess 


1 


SEXLOVB 


7.738 


2.623 


-.169 


-.934 


2 


COMMIT 


2.873 


2.226 


1.461 


1.336 


3 


ALWAYTHR 


2 . 723 


2 . 358 


2 . 451 


1. 747 


4 


HURTHINK 


7 . 538 


2 . 488 


- . 710 


- . 643 


5 


ALREMIND 


4 . 104 


2.644 


- . 265 


. 790 


6 


MEMORIES 


2.142 


1.813 


5.712 


2 . 241 


7 


HARDWORK 


2 . 469 


2 . 314 


3.231 


1.970 


8 


ACTOWILL 


5.618 


3 . 129 


-1 . 271 


, 015 


9 


PERMFEEL 


5.885 


3 . 213 


-1. 377 


-.083 


10 


LUVPHYSI 


8.467 


2.204 


1.763 


-1. 548 


11 


NOTSENSE 


5.523 


3.023 


-1.161 


. 042 


12 


EFFORT 


3.508 


2.746 


-.263 


.942 


13 


NOEXPECT 


4 .712 


2.753 


-l.OOi 


. 312 


14 


FAITHFUL 


1.962 


1.919 


6 . 264 


2 . 566 


15 


AFFECTON 


2.065 


1.766 


7.313 


2 . 536 


16 


FIRSTSEE 


5.846 


2.828 


-1. 026 


- . 008 


17 


CONSUMIN 


7.023 


2.633 


-.807 


-.482 


18 


UNCONTRL 


5.512 


3 .168 


-1.371 


.036 


19 


WORLDCTR 


6.842 


2,797 


-.949 


-.480 


20 


ALLGOOD 


6.685 


2 .846 


-1.069 


-.401 


21 


LOOKS 


8.469 


2.155 


1.912 


-1. 587 


22 


TIMEFAST 


4.762 


2.972 


-.987 


. 401 


23 


TOTALIVE 


3 .185 


2 . 385 


1.178 


1.333 


24 


NOLAST 


6 .777 


3 . 154 ' 


-1 .059 


- . 591 


25 


SUMCONSU 


3 . 496 


2.329 


. 365 


.888 


26 


ALIVE 


4.481 


2.957 


-.767 


.625 


27 


AFRAID 


7.277 


2.664 


-.648 


-.670 


28 


CBEAUTY 


3 .408 


2.383 


.604 


1.066 


29 


DESPERAT 


3 .896 


2.663 


-.273 


.790 


30 


CONSTANT 


4.756 


2.830 


-.880 


.463 


31 


ALUNIQUE 


1.965 


1.804 


8.441 


2.781 


32 


INATTENT 


7 . 358 


2 . 613 


-.257 


- .820 


33 


SEEMEANG 


3 . 765 


2 .317 


. 670 


.904 


34 


LESSSIGN 


5.346 


2 .826 


-1 . 095 


. 277 


35 


LUVBLIND 


4 .745 


2.864 


-.902 


. 500 


36 


SEXESSEN 


5 . 677 


3 . 099 


-1.331 


. 075 


37 


PICKPHYS 


3.396 


2 . 303 


. 850 


1 . 135 


38 


FORGIVE 


2.673 


1.838 


2 .349 


1 . 433 


39 


ONLYFEW 


3.162 


2 . 390 


1.081 


1 .324 


40 


NOTREALI 


5.231 


2 . 607 


-.766 


. 194 


41 


SAYANYTH 


3.581 


2 .810 


.054 


1.076 


42 


THOUGHT 


5.761 


2.903 


-1.206 


.027 


43 


LLREMIND 


6 .327 


2.860 


-1.133 


-.236 


44 


FIRSTSIG 


6.650 


3 .664 


-1. 485 


-.477 


45 


EMOTIONL 


2.798 


1.857 


1.723 


1.249 


46 


DAYDREAM 


6 .973 


2 .715 


-.843 


-.484 


47 


HAPPYSAD 


5. 510 


2.874 


-1.015 


.101 


48 


REJECT 10 


6 .308 


2.728 


-.964 


-.235 


49 


SELFSTUF 


3.742 


2.311 


. 317 


.884 


50 


NOCONTRL 


5.685 


2 .783 


-1.031 


.052 



15 

18 



51 UNCRTAIN 5.927 

52 TOTALLUV 5.177 

53 CONCENTR 7.058 

54 WORRY ATT 5.754 

55 FEARREJC 5.421 



2.806 -1.072 -.066 

3.071 -1.202 .239 

2.701 -.836 -.541 

3.002 -1.25S .003 

2.826 -1.042 .076 



ERIC 



16 

Jf9 



Table 2 

Factor structure from Correlation Matrix 



1 


SBXLOVE 


0 


.160 


0 


. 010 


0 


.010 


0 


.069 


-0 


.086 


0 


. 530 


~0 


.052 


0 


. 169 


2 


COMMIT 


0 


.176 


-0 


. 069 


0 


. 553 


0 


9 105 


-0 


.056 


-0 


. 063 


-0 


. 209 


0 


. 105 


o 




0.117 


u 


1 n '5 

. XU J 


u 


• 0 X o 


u 


ICC 
• XO o 


u 


n n n 


U 


n n o 
. UU Z 


U 




u 


O O '3 
. U U 0 


A 


HlIRTHTNIf 


0 


.455 




06R 

m \i yj \j 


n 


1 Rfi 

^ X «^ o 


0 


29 7 


0 


005 


0 


187 


0 

yJ 


059 

9 yJ mj J 


0 


16R 

9 AmSJ 'J 


'J 


fc\ iSj w X 


0 


-608 


0 


. X o o 


n 


. ^ 7 


0 


203 


0 


007 


-0 


1 '^4 
. X ^ ^ 


0 

yj 




-0 


074 

9 yJ t ^ 


6 


MEMORIES 


-0 


.023 


0 


304 


0 


576 


0 

\j 


034 


0 


018 


-0 


110 

. X X ^/ 


0 


066 
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18 



o 21 

ERIC 



Table 3 

Factor Structure from Variance-Covar iance Matrix 
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Appendix c 
Residual Matrix 
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